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ABSTRACT 

This document contains three poster presentations from a 
conference on human resource development. "Intrinsic and Extrinsic Work 
Motivators: Implications for the Incoming Air Force Officer 

Workforce" (Stephanie K. Johnson, Jason J. Davis, Christopher Rate) reports 
on a study that explored the literature relating to work motivators to find 
Air Force specific data on intrinsic and extrinsic work motivators and use 
that data to investigate the occupational selection hypothesis and 
occupational socialization hypothesis and their relation to retention in the 
Air Force. "MBTI (Meyers-Briggs Type Indicator): A Tool to Enhance Team 
Learning" (Brenda S. Gardner, Sharon J. Korth) describes how the Meyers 
Briggs Type Indicator can be used to enhance team learning in a graduate- 
level human resource development program. "Using Technology to Help Foster 
Critical Thinking and Reflection in Distance and Classroom Instruction: A 
Poster Presentation of the 1 R9 1 Process" (David Ripley) describes the 
development and application of the R9 process, which is an original 
pedagogical strategy to increase critical thinking and reflection in domain 
knowledge based courses in distance education and face-to-face classroom 
settings. The paper discusses selected R9 application issues and presents an 
example illustrating the process activities, venues, and instructor roles 
involved using the process in a typical class. All three papers contain 
substantial bibliographies. (MN) 
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Intrinsic and Extrinsic Work Motivators: Implications for the 
Incoming Air Force Officer Workforce 

Stephanie K . Johnson 

Jason J. Davis 

Christopher Rate 

United States A ir Force A cademy 

The purpose of this study is to explore the literature relating to work motivators and to find Air Force 
specific data on intrinsic and extrinsic work motivators . It investigates both the occupational selection 
hypothesis and the occupational socialization hypothesis, and relates them to retention in the Air Force . 

The literature and study results note that there is more research to be done with work motivators . 

Key Words: Intrinsic, Extrinsic, Work motivators 

To be a cutting-edge business, employers need to be cognizant of the changing nature of the workforce. Employers, 
like the United States Air Force, are investing in training and recruitment while realizing that a return on that 
investment requires an understanding of what motivates their workforce, both intrinsically and extrinsically. A 
strategic source of information to both employee and employer should be the intrinsic and extrinsic motivators that 
are deemed important by the workforce. The purpose of this research was to explore the issues surrounding intrinsic 
and extrinsic work motivators. Such accurate information allows for the development of a unique operating model 
that mirrors the existing reality and will help to benefit both the employee and the employer. 

Extrinsic motivators are rewards derived from the job but external to the work itself such as income, prestige, 
and security. In contrast, intrinsic motivators include rewards obtained directly from work experience such as 
interest, challenge, responsibility, autonomy, and other similar gratifying features. 

To explore these issues four enabling research questions were developed: 

1 . What are the expectations of incoming Air Force officers regarding both intrinsic and extrinsic work motivators? 

2. How will expectations of work motivators, or value placed on specific intrinsic and extrinsic work motivators, 
change as Air Force officers continue throughout their career? 

3. Are gender differences significant in the value placed on various intrinsic and extrinsic work motivators? 

4. What are the socio demographic variables associated with new entrants that may account for some of the intrinsic 
and extrinsic workplace motivators expected by incoming officers? 

It was the aim of this research to begin with a small population, incoming Air Force officers, and use their 
preferences to help further the knowledge of intrinsic and extrinsic work motivators. It is first meaningful to 
understand the characteristics of the new workforce generation as it provides a foundation from which they form 
these work motivators. 

The Air Force Newcomers th 

Strauss and Howe (1997) describe the incoming workforce generation (bom between 1961 and 1981) as the 13 
Generation. This generation, also called “Generation X,” comprises a large part of the Air Force, with approximately 
70 percent of the men and women being age 34 or younger. It is assumed that strategically aligning intrinsic and 
extrinsic work motivators for this generation should pay dividends in the form of retention and recruitment. 

Socialization influences such as parents, peers, media, education, religion, technology and culture, can guide 
how this generation are intrinsically and extrinsically motivated in the work environment (Burley, 1994). For 
example, Izzi (1998) describes fast-food dinners, over-scheduling and high-tech bedrooms as common occurrences 
in socializing the “13 lh generation.” Also, caring tasks such as childcare, elder care, lawn care, paying bills, buying 
groceries, washing cars, and home chores (among others) are being accomplished by hired help. In the work 
environment Tulgan (1995) suggests that Generation X was raised by both parents working loyally for one 
company, succeeding and then experiencing a downsizing, a restructuring or a buy-out. This mistrust of loyalty has 
brought in a generation of workers who value independence and an entrepreneurial spirit. Tulgan (1995) links their 
short attention spans to their environment of fast-paced technology and the information age. Specific to the Air 
Force, Omicinski (1999) highlights that the new generation of Air Force officers is more likely to come from fathers 
with no military experience to pass along to their sons or daughters. 

Copyright © 2002 Stephanie K. Johnson, Jason J . Davis and Christopher Rate 
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The generation of workers preparing to enter the workforce in the next few years may have little real-world 
experience with work issues. Furthermore, their expectations of work motivators may be contradictory (Johnson, 
2001). It is suggested that a learning model be developed that will educate and further align expectations with the 
reality of work issues. 

Formation of Work Values 

The first step in designing a learning model is to find out where new workers’ values lie with regards to expectations 
in the workplace. As Mortimer and Lorence (1979) noted over twenty years ago: “First, people choose occupations, 
and thus select their work experiences, on the basis of their values. At the time of entry into the labor force, the 
individual attempts to select a work role that will maximize rewards deemed most important . . (p. 1364). More 

recently, Johnson (2001) notes “in the context of work, work values are beliefs about the desirability of various 
work features and are usually defined by referencing several types of rewards derived from working’’ (p. 317). 

The research on work values reveals two competing ideas. The first is the “occupational selection hypothesis, 
in which people choose their occupations based on psychological characteristics and values already in place 
(Mortimer & Lorence, 1979; Lindsay & Knox, 1984; Johnson, 2001). The second is based on the reinforcement 
hypothesis, or what Mortimer and Lorence (1979) called the “occupational socialization hypothesis” which asserts 
that rewards obtained on the job grow in importance over time and mold work values, whereas the absence of 
rewards lessens their values over time (Mortimer & Lorence, 1979; Johnson, 2001). This idea of value 
reinforcement through an occupation is consistent with Rokeach’s (1973) theory of values, which affirms that values 
will change to preserve one’s positive self-concept, and likewise valuing something that one does not have could 
diminish self-esteem (Rokeach, 1973). 

Another issue to consider is that values formed in early adulthood are likely to change with work experience 
and age. Johnson (2001) argues that adolescent occupational aspirations exceed what is available in the labor 
market, making their work values reflect strong desires for work motivators that are unrealistic (Johnson, 2001, p. 
316). Her longitudinal study found that the proportion of respondents rating each job feature as “very important” 
declined over the 13-14 year gap between testing phases. The question is whether or not you can accurately 
determine work values at the high school or college level that will be general izeable to adult worker expectations. 
Johnson (2001) notes that as young people gain work experience, they also gain a better understanding of the labor 
market and what is realistically attainable. 

It is also necessary to address a possible gender difference with regard to work motivators. Studies have found 
that women place a greater emphasis on intrinsic, altruistic, and social rewards from work, while men place more 
importance on the extrinsic motivators obtained from work (Johnson, 2001; Lindsay & Knox, 1984). Johnson also 
found that the gender gap in extrinsic values did not lessen throughout the time between testing yet both males and 
females attached less importance to extrinsic values over time. As times change, this gender differentiation due to 
socialization is decreasing. Women are beginning to close the gap regarding the value they place on extrinsic 
motivators while maintaining a stronger orientation towards intrinsic, altruistic, and social rewards (Johnson, 2001). 
In support of the reinforcement hypothesis, as young people begin to become more realistic about what rewards are 
available to them in the workplace, the gender difference in work values may narrow (Johnson, 2001). 

Another variable is sociological changes, particularly family roles and economic standing. Johnson (2001) 
describes a gender model that would predict weaker effects of work motivators for women when compared to men, 
with women experiencing a decrease in the importance of work motivators as they enter family roles. Additionally, 
family roles for males solidify their breadwinner role increasing their focus on extrinsic motivators. In general, 
having a family and children has been positively associated with an increase of importance placed on extrinsic 
motivators across both genders (Johnson, 2001). Lindsay and Knox (1984) assert that social structure variables on 
life changes, particularly parenthood, education, origin, race, and gender, have a big impact on the formation of 
work values. 

Economic status also plays a significant role in facilitating the formation of work values. Mortimer and Lorence 
(1979) note that when an individual has inadequate economic resources, income and other extrinsic motivators 
assume a higher priority. Comparatively, when material needs are met the salience of extrinsic motivators lessens 
and the worker concentrates on more intrinsic job satisfactions (Mortimer & Lorence, 1979). Furthermore, intrinsic 
occupational rewards increase with occupational prestige, while extrinsic rewards are highly valued at lower 
occupational levels (Mortimer & Lorence, 1979). 

In summary, significant trends exist in the study of work values. However, numerous studies provide support 
for different hypotheses, or conflicting findings with regards to gender differences, the stability of work values over 
time, and the impact of demographic and socioeconomic conditions on work values. This study seeks to identify 



what work values officer candidates consider most important, the possible gender differences across these values, 
and the possible incongruity between valued mo tivators and current motivators that may be effecting retention rates 
in the United States Air Force. Most importantly, the study will seek to identify whether incoming Air Force officers 
will value extrinsic work motivators over intrinsic work motivators, and whether there will be any gender difference 
in this preference. 

Method 

A survey was designed to provide insight into the different values placed on both intrinsic and extrinsic work 
motivators. The data helped analyze how cadets from the United States Air Force Academy (USAFA) and various 
Reserve Officer Training Corps (ROTC) programs preparing to enter active duty perceived intrinsic and extrinsic 
work motivators. Comparisons were made between commissioning source and gender. Furthermore, in addition to 
the previously presented research questions, two hypotheses were formulated: 

Hypothesis 1 : There will be no difference between incoming USAFA and ROTC officers in the respect that both 
will value extrinsic motivators more than intrinsic motivators. 

Hypothesis 2: Extrinsic motivators will be rated as more important than intrinsic motivators across genders. 

Sample 

The sample consisted of 343 cadets entering the active duty Air Force within the next year. Specifically, the 
sample contained 275 USAFA cadets and 68 ROTC cadets enrolled at universities in Colorado and Wyoming. The 
average age was between 21 and 22 years old. The majority of the sample had mothers with some college or trade 
school education that worked either full or part time while the cadet had lived at home. The majority of the fathers 
of the sample population had a college degree. Of the ROTC population, nine were married with the majority having 
a spouse that had some college or trade school education that worked full-time. Of the nine married cadets, six had 
children. 

Instrument 

An existing multipart survey, The Career Issues Survey, was directly administered for this study (Covin & 
Brush, 1993, 1991). The instrument language was slightly modified to fit the cadet population and context. For this 
study, only sections two and three of the Career Issues Survey were used (questions 57-91). These two sections 
contained questions related to extrinsic and intrinsic motivators. The participants completed the surveys in a 
classroom setting with the researcher present to answer questions. 

Data Analysis 

To assess for “dimensionality and commonality” (Covin & Brush, 1993, p. 36) among the 34 Likert-scaled 
items, a factor analysis was performed, aligning the questions into two factors - intrinsic and extrinsic motivators. 
The factor analysis was used to generate subscores for the two dependent variables - intrinsic and extrinsic work 
motivators. Next, the subscales derived by the factor analysis were utilized to compare commissioning source and 
gender. Once the researcher established the reliability of the factors derived by the use of a reliability analysis 
function, a two-way Analysis of Variance (GLM Univariate) and a Multivariate ANOVA were used to test for 
interactions between the two independent variables (commissioning source and gender) on both Factor One 
(intrinsic motivators) and Factor Two (extrinsic motivators). 

Factor Analysis 

The first step in the data analysis was to identify what factors would be present in the data. This was 
accomplished using a factor analysis. Fraenkel and Wallen (1996) describe factor analysis as “a technique that 
allows the researcher to determine if many variables can be described by a few factors... the technique essentially 
involves a search for clusters of variables, all of which are correlated with each other” (p.3 14). A factor analysis was 
performed using principal component factoring and a varimax rotation using questions 57-91 of the survey and 
extracting all of the questions into two factors. The questions within each factor with loadings above .5 were 
identified and retained (see Appendix). 

To ensure that the loading in each factor was actually contributing to the factor, a reliability analysis was run. By 
analyzing the alpha reliability coefficient, one can determine the strength of the scale. For Factor One, alpha was 
.84, and for Factor Two alpha was .79. In both Factors, the alpha reliability coefficient would not have improved 
with a removal of any one item. 
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A test for correlation amongst the two factors was also accomplished. The purpose was to verify whether a 
relationship existed between the two factors. A Pearson product-moment coefficient (r) was calculated for the two 
factors. Correlation coefficients for Factor One and Two were .17, revealing no significant correlation between the 
two factors. 

Factor One. 

Factor One contained questions directed at weighing the importance of intrinsic motivators to incoming Air 
Force officers. After analyzing the factor loadings and fie descriptors present in each question, it was decided to 
retain questions 72, 74, 78, 79, 81, 84, 85, 88, 89, 90, and 91. Questions 63, 86, and 87 were removed because their 
descriptors did not fit with the other questions. 

A high score on any item in the factor would indicate that the particular intrinsic motivator is a very important 
aspect of the workplace to the sample. There was no significance in commissioning source, gender, or the 
interaction for Factor One. The data demonstrates that intrinsic motivators were rated higher than extrinsic across 
both commissioning sources and gender (see Tables 1 and 2). The men rated the importance of intrinsic motivators 
.02 higher than the women. In addition, ROTC cadets rated the importance of intrinsic motivators .01 higher than 
USAFA cadets. Furthermore, there was no significance in the univariate ANOVA performed on Factor One (see 
Table 3). 

Factor Two. 

Factor Two consisted of questions directed at weighing the importance of extrinsic motivators to incoming Air 
Force Officers. After analyzing the factor loadings and the descriptors present in each question in this factor, it was 
decided to retain questions 57, 58, 60, 64, 66, 67, 68, and 70. Questions 71, 80, and 82 were removed because their 
descriptors did not fit in with the other questions. 

For this sample, a high score indicated that certain extrinsic motivators are important aspects of the workplace. 
Factor Two had a lower mean than Factor One. The female sample’s mean was .34 higher than the males on this 
factor, which is indicated by the significance on gender in Factor Two: F (1,343)=6.74, p=. 01 (see Table 4). In 
addition, USAFA cadets* mean was .16 higher than ROTC’s, as indicated by the significance for commissioning 
source: F (1,343)=6.30, p=. 01. Therefore, both gender and commissioning source main effects were significant for 
Factor Two. 

Analysis for Research Questions and Hypotheses 

The research questions and hypotheses suggest that source of commission and gender will relate to intrinsic and 
extrinsic motivators. A two-way Multivariate Analysis of Variance (MANOVA) was used to test for main effects of 
the sample groups (USAFA cadets versus ROTC cadets, and men versus women) on the two factors. The 
MANOVA indicated there are significant main effects for both commissioning source (Wilks’s lambda = .97, F = 
4.61, df = 2/326, p = .01 1) and gender (Wilks’s lambda = .98, F = 3.22, df = 2/326, p = .041). The MANOVA also 
indicated no significant interactions between commissioning source and gender (Wilks’s lambda = .99, F = 2.31, 
df = 2/32, p = .10). 

The univariate tests for the influence of commissioning source and gender were significant (p<. 05) for the main 
effects of gender and commissioning source for Factor Two (extrinsic motivators). The interaction between gender 
and commissioning source was not significant in either Factor, nor was anything else significant in Factor One 
(intrinsic motivators). 

T-Test 

The purpose of paired-samples T-test is to compare the means of two variables for a single group. Since one of 
the objectives was to determine if incoming Air Force officers would value extrinsic motivators (Factor Two) more 
than intrinsic motivators (Factor One), a paired-samples T-test would reveal any difference. The mean for intrinsic 
motivators was 4.17 while the mean for extrinsic motivators was 3.69. This difference was significant on a two 
tailed measure, p = .000. 
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Individual Question Means 

Another analysis component is individual question means. These means reveal on a smaller level what the 
extremes are for importance or what is really not important to the incoming officer within the confines of both 
extrinsic and intrinsic motivators. The questions rated the most important included: 1) is intellectually stimulating, 
2) provides a feeling of accomplishment, and 3) medical benefits. These highest scores are inclusive of 
commissioning sources and genders. 

Interpretation 

The first research question asked what the expectations of incoming Air Force officers would be regarding both 
intrinsic and extrinsic work motivators. A descriptive analysis of means suggests the six extrinsic and intrinsic 
motivators rated the highest were: 1) Working Conditions (extrinsic), 2) Medical Benefits (extrinsic), 3) Allows 
adequate time for family and friends (intrinsic), 4) Provides job security (intrinsic), 5) Is intellectually stimulating 
(intrinsic), and 6) Provides a feeling of accomplishment (intrinsic). Conversely, the three questions rated lowest by 
the group, related to Factor Two, were 1) Childcare policies, 2) Maternity Leave Policies, and 3) Paternity Leave 
Policies (all extrinsic factors). 

Since the majority of 20-21 years olds do not have immediate concerns regarding children, and USAFA cadets 
are prohibited from having children while they are cadets, it is not surprising that the group rated these three 
questions lowest. However, the more interesting finding lies in the higher rated questions. The general theme of the 
highly rated questions was one of internal satisfaction due to accomplishment and the work environment, rather than 
one based on extrinsic motivators such as money. Although this finding generally does not support our hypothesis, it 
speaks volumes for those entering the Air Force because it shows that they value the nature of their work, and not 
their pay. This is a positive consideration as most Air Force personnel have the potential to earn more income in the 
civilian sector. Therefore, these findings suggest that the Air Force is sufficiently tending to these needs. With 
regards to retention, it would be important to focus on the motivators, both extrinsic and intrinsic, and to try to 
maximize those in the Air Force officer work environment. 

The second research question asked how expectations of work motivators, or value placed on specific intrinsic 
and extrinsic work motivators, would change as Air Force officers continue throughout their career. The data 
suggests that there is more importance placed on intrinsic motivators in the vorkplace rather than extrinsic 
motivators as can be explained by the difference in means and the paired-sample T-test. However, the first research 
question should be a good indication that it is a distinct possibility that the answer lies much deeper than a simple 
diagnosis. The answer may very well be that incoming officers value a distinct mix of intrinsic and extrinsic 
motivators based on individual socialization and environmental factors. In addition, without the use of a 
longitudinal study of incoming officers, both before entry and after a specified time in service, it is indeterminable 
how these values will change as their career progresses. However, the retention issues surrounding this question can 
be answered by the research. One of two things most likely happens. The first option is that incoming officers 
follow the “occupational selection hypothesis,” in which people choose their occupations based on psychological 
characteristics and values already in place (Mortimer & Lorence, 1979; Lindsay & Knox, 1984; Johnson, 2001). Or 
they follow the “occupational socialization hypothesis” which asserts that rewards obtained on the job grow in 
importance over time and mold work values, whereas the absence of rewards lessens their values over time 
(Mortimer & Lorence, 1979; Johnson, 2001). The thought is that if they are following the “occupational selection 
hypothesis” then retention solutions are minimal because they are basing their job preference off of predetermined 
values. Therefore, if the Air Force occupation does not satisfy those values then the officer will most likely separate 
from the Air Force. However, if it is the case that they are following the “occupational socialization hypothesis,” 
then it is important to find out what motivators are important to them as their career develops, and to positively 
reinforce those motivators for retention purposes. 

The third research question sought to identify gender value differences on various intrinsic and extrinsic work 
motivators. The data does not support gender differences related to intrinsic motivators; the gender main effect was 
significant for extrinsic motivators (Factor Two). One might argue that the differences in values placed on the 
extrinsic motivator questions are most likely due to different gender roles and the different value systems across 
genders. This finding warrants further research. As of now, data suggest no significant motivators that will 
drastically impact retention for one gender that would not also have a positive impact on the other. The fact that 
male rated extrinsic motivators as slightly higher than females is in accordance with past research (Mortimer & 
Lorence, 1979; Lindsay & Knox, 1984; Johnson, 2001). 

The fourth research question asked if there were any sociodemographic variables associated with new entrants 
that may account for some of the intrinsic and extrinsic workplace motivators expected to be delivered by their 
employer. It can be assumed from the data that cadets planning to enter the workforce are expecting some type of 
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assistance from their employer with issues related to work and family. An interesting demographic finding was that 
the majority of mothers of the cadets had worked either full or part-time while the cadet had lived at home. 
Additionally, all of the spouses of the ROTC cadets had jobs. This is an indicator that issues of work- family balance 
had been present during their formative years in which both of their parents worked. 

In summary, although the study answered the research questions quite well, it rejected both of the hypotheses. 
Intrinsic motivators were in fact rated higher than extrinsic motivators, which is the exact opposite of the prediction. 
The first hypothesis predicted that there would be no difference between USAFA and ROTC incoming officers in 
the respect that both will value extrinsic motivators more than intrinsic motivators. Although there was no 
significant difference between USAFA and ROTC on their ratings of intrinsic motivators (Factor One), 
commissioning source was significant for extrinsic motivators (Factor Two). In addition, both groups rated intrinsic 
motivators higher than extrinsic motivators. The second hypothesis stated that extrinsic motivators would be rated as 
more important than intrinsic motivators across both genders. Again, since intrinsic motivators were rated as more 
important across both commissioning source and gender, this hypothesis is also rejected. 

A major objective of this research has been to ascertain work motivators of incoming officers into the active Air 
Force. The efforts have resulted in data that can help assist with recruiting, retention and education. Additionally, the 
data begins to lay the foundations for future development in the area of intrinsic and extrinsic workforce desires. 
Toward this goal the research yielded results which indicate that incoming Air Force officers can differ in their work 
values. Further studies of work motivators, especially specific to Air Force officers are needed. A longitudinal study 
would reveal the changing nature of work motivators that would immensely help form a better learning model for 
the United States Air Force and other employers. Ultimately, this could increase retention and make for a more 
reliable and efficient workforce. 
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Appendix 

Factor One (intrinsic motivators) includes questions: 

72. Requires originality and creativity 

74. Encourages continued development of knowledge and skills 

78. Gives you the responsibilities for taking risks 

79. Requires working on problems of central importance to the organization 

8 1 . Provides change and variety in duties and activities 
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84. Permits advancement to higher levels of responsibility 

85. Permits working independently 

88. Is intellectually stimulating 

89. Permits you to work for superiors you admire and respect 

90. Permits you to develop your own methods of doing the work 

9 1 . Provides a feeling of accomplishment 

Factor Two (extrinsic motivators) includes questions: 

57. Level of Pay 

58. Child Care Policies 

60. Vacation policies 

64. Working hours 

66. Medical benefits 

67. Maternity leave policies 

68. Yearly pay increases 

70. Paternity leave policies 

7 1 . Allows adequate time for family/friends 

80. Provides ample leisure time off the job 

8 1 . Provides comfortable working conditions 

Note: For an entire copy of the survey refer to Covin & Brush, 1993 & 1991 



Table 1 . Mean Factor Scores of Male and Female Respondents 



Factor 


Men (n= 255) 


Women (n = 88) 


Factor 1 

Intrinsic Motivators 


Mean = 4.17 


Mean = 4.15 


Factor 2 

Extrinsic Motivators 


Mean = 3.61 


Mean = 3.95 



Legend of Likert Scale: Range was 1-5 with 1: Not Important, 3: Neutral, 5:Very Important 



Table 2. Mean Factor Scores ofUSAFA and ROTC Respondents 



Factor 


ROTC (n = 68) 


USAFA (n = 275) 


Factor 1 

Intrinsic Motivators 


Mean = 4.17 


Mean = 4.16 


Factor 2 

Extrinsic Motivators 


Mean = 3.57 


Mean = 3.73 



Legend of Likert Scale: Range was 1-5 with 1: Not Important, 3: Neutral, 5:Very Important 



ISource 


Sum of Squares 


df 


Mean Square 


t 


SIS: 


[Commissioning Source (Univer2) 


.04 


1 


.04 


\a55_ 


.694 


bender 


.164 


1 


.164 1 


697 


.404 


Hjniver2 * Gender 


.188 


1 


.188 ! 


|.801 


.372 


[Error 


79.291 


337 


.235 


h— 


- 


uotal 


5995.736 


341 




t 





Note: p<. 05 
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Source 


Sum of Squares 


df 


Mean Square 


F 





Commissioning Source (Univer2) 


2.23 


1 


2.23 


6.30 


.013* 


Gender 


2.39 


1 


2.39 


6.74 


.010* 


Univer2 * Gender 


1.09 


1 


1.09 


3.07 


.081 


Error 


120.14 


339 


.354 


- 


- 


Total 


4820.42 


343 


- 


- 






P-1 



MBTI: A Tool to Enhance Team Learning 



Brenda S. Gardner 
Sharon J. Korth 
Xavier University 

Using teams of employees to get work done in organizations is becoming increasingly more common across 
the world. This article focuses on an approach to training HRD professionals to learn and model effective 
team behavior. It describes how the MBTI is used as a tool to enhance team learning in a graduate level 
HRD program. Conclusions and recommendations for HRD academic and training programs are explored. 

Keywords: Teams, Learning Styles, Academic Programs 

Across the world, organizations utilize teams in the workplace. From the United Kingdom, Australia, Ireland, Malaysia, 
and the United States, descriptions of workplace teams abound (Wellins, Byham & Dixon, 1994). While some of these 
teams are domestic, with all of the members from the same country, the increasingly global workplace has led to the 
creation of many transnational teams as well. The number of teams used in the United States has increased since the 
1980s when total quality and other similar management philosophies pointed to team-based work environments as the 
success formula of the future (Dobyns & Crawford-Mason, 1991). When faced with the challenges of a changing 
workforce, global competition, technological innovations, decreases in productivity, and shifting work values, many U.S. 
organizations used teams as part of their newly designed work structure. 

Problem Statement 

HRD professionals are often responsible for the training of teams in the workplace and must understand the elements of 
team learning and effective team behavior. The best learning comes from personal experience, so the challenge is to 
provide HRD professionals with team learning experiences that they can transfer to the workplace. 

Theoretical Framework 

The Myers -Briggs Type Indicator has been used extensively in organizations and educational institutions and is based 
on the work of the Swiss psychotherapist Carl Jung (Myers, 1993; Provost & Anchors, 1987). The MBTI reflects 
individual preferences for energy (Extraversion and Introversion), information gathering (Sensing and Intuition), decision 
making (Thinking and Feeling), and lifestyle (Judging and Perceiving). Each of these four dimensions is measured on a 
scale; sixteen unique personality types result from the combination of the four MBTI preference scales. 

One common use of the MBTI is with learning style research. For example, extraversion has been found to be 
associated with preferences for working in a group, whereas introversion is related to working individually. Sensing is 
associated with practical interests, while intuition is related to tasks that call for imagination. Research has shown that 
students high in the thinking dimension prefer objective material to study, while those high in the feeling dimension learn 
through personal relationships. Finally, the judging component relates to formalized instruction, whereas the perceiving 
component corresponds to informal problem solving (Lawrence, 1984). 

Lynch & Sellers (1996) developed descriptions of four college/leaming environments based on a quadrant 
combination of MBTI attitude and perception preferences which were most applicable to learning (1996) (See Figure 1). 
Lawrence (1994), and Lynch and Sellers use combinations of the first two dimensions of the MBTI to describe different 
types of classrooms. 
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Figure L Learning Style Preferences and Training 



ES Training Activities 
Extravert- Sensing 

- facts, information 

- organized schedule 

- practical hands-on experience 

- field trips 

- on-the-job training 

- working in groups 

- thinking aloud 

- group reports 

EN Training Activities 
Extravert - Intuitive 

- variety 

- group work on problem-solving tasks 

- discuss situations to spark imagination 

- freedom to choose alternatives 

- deadlines and strict time schedule de-emphasized 



IS Training Activities 
Introvert - Sensing 

- practical, realistic 

- organized with steady pace 

- step-by-step sequence, attention to detail 

- clear expectation from trainer 

- allow time to think before working in groups 

- demonstrations 

- computer assisted instruction 

- films, videos, audio-visual aids 



IN Training Activities 
Introvert - Intuitive 

- serious reading 

- tutorials, independent study 

- autonomy in setting own goals and plan 

- allow time to think through topic 

- explore special interest topics in depth 

- move at the rate preferred by the students 

- allow for individual work 



The MBTI has also been used extensively to enhance teamwork. According to Hirsh, the MBTI specifically aids 
team members by: 

• reducing unproductive work 

• identifying areas of strength and possible area of weakness for the team 

• clarifying team behavior 

• helping to match specific task assignments with team members according to their MBTI preferences 

• supplying a framework in which team members can understand and better handle conflict 

• helping individuals understand how different perspectives and methods can lead to effective problem 
solving 

• maximizing a team's diversity in order to reach more useful and insightful conclusions. (Hirsh, 1992, p. 7). 
Premise and Research Question 

This research is based upon the premise that (a) teams are common in today's workplace; (b) HRD professionals must 
learn and model effective team behavior; and (c) academic and training programs must be developed that teach and model 
team behavior. The cycle of learning and performance starts with academic/training programs teaching tearmdated skills 
to HRD professionals, continues as HRD professionals utilize these skills on their jobs as they train others, and leads to 
improved team skills for employees. 

The following research question served as a guide for this study: How can the MBTI be used as a tool to enhance 
team learning in academic settings? 

Methodology 

In this study, quantitative and qualitative data were collected in a course on organization behavior, offered during the first 
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semester of graduate study at a private US university, as part of an academic program in HRD. To determine personality 
and learning type preferences of HRD professionals, data were collected from a total of 241 HRD graduate students over a 
six-year period enrolled in this course, and were analyzed using Statistical Package for Social Sciences (SPSS) for 
Windows Release 6.1. 

The students were all working adults, ranging in age from 21 to 56, with a mean age of 37, standard deviation of 7.98. 
Seventy -six percent of the group was female, and years of full-time work experience ranged from 0 to 37, with a mean of 
13.2 and a standard deviation of 8.02. During the second meeting of the organization behavior class, the Myers-Briggs 
Type Indicator-Form G (MBTI), developed by Katharine Cook Briggs and Isabel Briggs Myers, was administered to the 
students by the researchers who were also the professors of the course. 

In addition, qualitative data were collected through participant-observation and document review. Documents 
included course materials, assignments, as well as student learning journals and reflective papers. Data were analyzed by 
constant comparative method (Lincoln & Guba, 1985), reflecting on the meaning of what was heard and seen, and using 
existing literature when useful. 

Results and Findings 

MBTI Personality Preferences 

This study found that in this HRD graduate student/practitioner group, the majority of students were Extravert, 
Intuitive, Feeling and Judging. This contrasts with the national US adult norms of majority preferences in Introvert, 
Sensing, and Thinking (Hammer & Mitchell, 1996). Preferences of other related fields are also included in Table 1, and 
indicate similarities with the HRD group in the judging preference (Myers & McCaulley, 1985). 



Table!. MBTI Personality Preferences 





HRD Graduate 
Students 
(n=241) 


US Adult Norms 
(n= 1,267) 


Personnel & Labor 
Relations Workers 
(n=90) 


Employment 

Development 

Specialists 

(n=80) 


Teachers - 
University 
(n=2,282) 


Extravert 


61% 


46% 


54% 


54% 


46% 


Introvert 


39% 


54% 


46% 


46% 


54% 


Sensing 


43% 


68% 


56% 


40% 


36% 


Intuition 


57% 


32% 


44% 


60% 


64% 


Thinking 


49% 


53% 


66% 


54% 


53% 


Feeling 


51% 


47% 


34% 


46% 


47% 


Judging 


61% 


58% 


67% 


61% 


66% 


Perceiving 


39% 


42% 


33% 


39% 


34% 



Use of MBTI to Design Team Training 

Gathering data about personality and learning style preferences in relation to energy, information gathering, decision 
making, and lifestyle provided researchers with a layer of analysis useful in the design and implementation of team 
training in the classroom, as well as a tool that these students could use in their own teams in the classroom and back on- 
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the-job. 

Researchers worked with students to help them understand how preferences could aid in improving their classroom 
groups, as well as the teams they were implementing and leading in the workplace. After reviewing and processing Jung's 
personality type theory and its relationship to the students’ MBTI results, researchers organized students into small 
groups of 6-7 for group projects lasting over two semesters. Teams were organized, however, not by any grouping of 
specific MBTI types, but toward a mixture of gender, race, and levels and type of work experience. 

Activities designed to help the students increase their self-awareness using MBTI personality preferences included, 
of course, the administration of the MBTI and thorough processing of its results. Researchers posed questions of the 
students such as: How do I prefer to gather information and make decisions? What leadership qualities do I value? How 
do I prefer to learn? What irritates me about my teammates? How can I contribute to the team? How do I react to 
conflict? How well do I listen? How do I give and receive feedback? What role do I play in a group setting? How well do 
I lead or follow? For example, students with Introversion-Sensing-Thinking-Judging (ISTJ) profiles are likely to be 
thorough, systematic, hard-working, and careful with detail. With this heightened self-knowledge, these students may 
better understand why they value traditional, hierarchical approaches to leadership and why they get irritated at those 
who do not follow through with commitments. Two project assignments specifically helped students increase their self- 
awareness: entries in a learning journal kept over two semesters, and a comprehensive analysis of the student's learning 
as a culminating project for the year. 

To assist students in applying personality type theory to groups and teamwork, researchers organized experiential 
activities for the teams which were processed using MBTI results. For example, problem-solving activities and outdoor 
leadership activities all provided data that could be analyzed by the students using personality type theory. Group 
projects culminating in a presentation, report and group analysis were natural areas in which group process questions 
were asked of the students: How do we manage our meetings? How do we make decisions or solve problems? How do 
we manage our projects? How do we develop shared goals and create/maintain commitment to those goals? How do we 
create a climate that fosters trust and openness? How do we balance utilizing the strengths of our members while 
fostering individual personal development? For example,"time" can become an issue with teams and their management of 
meetings. Team members with the Thinking preference may prefer to be brief, concise and task-oriented, while those with 
the Feeling preference may want to take the time needed to build and maintain interpersonal relationships. Once team 
members understand that these differences exist and recognize the value of both perspectives, they can leverage them to 
enhance the effectiveness of their team meetings. Again, the experiences and analysis using personality type theory 
aided in the students' understanding and improving their classroom group, as well as how to better lead and implement 
teams back in their workplaces. 

Course feedback from the students has been generally very positive. Informal comments, information provided on the 
written individual and group analysis papers, and formal end-of-class evaluation results indicate that the students find 
the MBTI a useful tool in increasing their self-awareness and enhancing their classroom and workplace group 
functioning. One student, in her individual analysis, explained that knowing about the different MBTI preferences 
helped her realize that she may have been misjudging the quiet members of her group as lacking commitment. As a result, 
she was able to appreciate their differences in style and help find ways to utilize their strengths. Another student 
reported about how he successfully used one of the experiential class activities with a group he was facilitating in the 
workplace. 

MBTI Learning Styles Consideration for Training 

The MBTI can also provide insight into learning style preferences of class participants. Because there will be a mix 
of styles in any group, it is important to create training that includes learning activities appealing to all preference types. 
Knowing the learning style preferences of the participants in a given group as well as the types of corresponding 
activities allows the flexibility to adapt the training to meet the specific needs of the group. 

Applying these ideas to team building training implies that there will be a variety of individual and group activities, 
and that some portions of the training will be practical, hands-on, and structured while other portions will allow for 
personal exploration. The intent is to utilize an assortment of methods so as to tap into current learning strengths as well 
as to stretch and increase the participants' learning abilities in other dimensions. Researchers utilized discussions, 
psychomotor activities (including outdoor experiential exercises, if possible) and group work to appeal to the extraversion 
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dimension. Conversely, reading, individual work, and time for internal processing were also included to support the 
learning strengths of those with introversion preferences. Some parts of the instruction, for example, on project 
management, progressed in a step-by-step manner (ex travert -sensing), while other parts involved having students find 
their own way in the material (introvert -intuition), such as in conducting research and writing papers. Finally, the course 
involved a mix of prescribed tasks, such as the development of an annotated bibliography on an HRD related topic 
(introvert -sensing), and discovery tasks, such as finding creative ways to solve hypothetical problems in a group 
(extravert-intuitive). 

Conclusions and Recommendations 

First, academics who design classroom training for HRD students may wish to use a tool such as the MBTI as a lens in 
which to review their instructional methods to determine balance and appropriateness for their group. The HRD graduate 
student sample reported in this article indicated that the least favored learning environment was Introvert -Intuitive, which 
was also reported in Lynch and Sellers' 1996 study. In a number of studies, however, Introvert -Intuitive is a very common 
personality style and learning preference for college professors (Lynch & Sellers; Murphy, 1987; Myers & McCaulley, 
1985). (See Table 1). If professors "teach to type” (create a learning environment consistent with their own learning style 
preferences), a mismatch may occur between learners' preferences and professors' activities. Awareness of the 
implications that may develop in the classroom as a result of this type of situation can be a first step for reviewing the 
curriculum design for a course. 

In contrast, the most favored learning style preference was Extravert-intuitive for HRD graduate students. HRD 
professionals who are designing classroom training in the workplace for other occupations such as engineers, health care 
professionals, computer analysts, etc., would also do well to determine if their own preferences for learning activities 
might be overshadowing the learning needs of their students. 

Finally, much more work must be done to study the role of the HRD professional in team learning and 
team/organizational effectiveness. This paper described how knowledge of the MBTI personality and learning styles can 
be used to design training for team members in the classroom. More research is needed to establish measures of team 
learning and transfer of skills back on the job. 

Clearly, the study described in this paper is just a beginning. Further research is needed in each phase of the team 
leaming/performance cycle (academic programs, team development in the workplace, team and organizational 
effectiveness) and to determine the connections between the phases (See Figure 2). 

Figure 2. Team Learning/Performance Cycle 



Teams in the Workplace 




HRD Professionals 
Modeling & Teaching 
Team Skills in Training 
Programs 




HRD Professionals 
Learning Team Skills in 
Academic Programs 



Contribution to the Field of HRD 



This paper showed how the MBTI can be used to enhance the team learning of HRD professionals in an academic 
program. In the workplace, HRD professionals have the primary responsibility for developing and implementing team- 



related training so that employees can develop the necessary competence. Further, they have an obligation to model 
effective team behaviors themselves. How do HRD professionals learn these inportant team skills? The team 
leaming/performance cycle starts with academic programs increasing their HRD students’ self-awareness and learning 
about groups and organizations. HRD professionals must take an active role in learning how to model and teach effective 
team behaviors so that teams are trained for success, the leaming/performance cycle continues, and organizations 
maximize their capabilities to transform in order to meet strategic goals. 
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Using Technology to Help Foster Critical Thinking and Reflection in 
Distance and Classroom Instruction: A Poster Presentation of the “R9” Process 

David Ripley 

The University of Canterbury 

This poster presentation describes the development and application of the "R9 " process, an original 
pedagogical strategy designed to increase critical thinking and reflection in domain knowledge-based 
courses, in both distance education and face-to-face classroom settings. Individual steps of the process are 
described and examples of its application are provided. 

Key Words: Critical Thinking, Reflection, Pedagogy 

The importance of critical thinking and reflection in education has been stressed repeatedly over the past several 
years, in national and international journal issues, numerous books and articles, and workshops held by various 
education authorities (Baron & Sternberg, 1987). Since this poster presentation is primarily about the R9 process 
and its use, there is no intent to provide a comprehensive review of the very broad range of critical thinking 
literature. However, critical thinking and reflection are discussed to the degree necessary to position the R9 process 
for the reader within that body of thought. It is important to note that the R9 process evolved from a desire to 
develop a pedagogical approach that would allow one to make critical thinking and reflection a part of the normal 
conduct of a domain knowledge-based course, rather than conferences, entire a desire to provide a course on critical 
thinking, such as that described by Mingers (2000). 

Theoretical Framework 

The literature on critical thinking and reflection covers a broad spectrum. Critical thinking, in particular, can be 
viewed from various perspectives. 

1. One can take the limited perspective of a highly focused problem-solving approach of a typical total quality 
management (TQM) manual. 

2. One can move to a broader - but still skill-based - approach that is more about the how and what of 
critical thinking. This would be more in keeping with the informal logic movement (Ennis, 1987; Kiersky 
& Caste, 1995; Quellmalz, 1987; McPeck, 1990). 

3. Beyond this can be found what one might term a more socially conscious perspective. To use Alvesson's 
and Willmott's (1992) term, a "softer" approach (p. 432) that shows some concern for higher-order human 
needs. 

4. One can also move toward positions that indicate the goal of critical thinking should be social 
emancipation. This latter position is more in keeping with the ideas of critical pedagogy aid critical 
practice (Mezirow, 1990; Freire, 1970), and moves toward critical theory (Habermas, 1994; Foucault, 
1994) and critical social science (Fay, 1987). 

On such a spectrum, the position of the R9 process is relatively modest. 1 wanted to provide something more 
than the basic problem solving of a TQM manual but something less than a call for emancipation or a study of 
critical social science. The process is closer to the informal logic movement than to the socially conscious 
perspective. I would like my position to be congruent with Said’s (1983) comment that “It is not practising criticism 
either to validate the status quo or to join up with a priestly caste of acolytes and dogmatic metaphysicians” (p. 5). 1 
am taking Said somewhat out of context - he was referring to literary criticism - but the thought warrants reflection 
in this context. In the context of the domain knowledge-based course, I want to concentrate on fundamental 
questions such as those raised by Hughes (1996). Are the arguments and propositions sound in a logical sense? Do 
the conclusions follow from the premises? Are the premises justifiable? What are the underlying assumptions? A 
good working definition for critical thinking for my purpose is that of Brookfield (in Mezirow, 1987), which states, 
“Being a critical thinker involves more than cognitive activities such as logical reasoning or scrutinizing arguments 
for assertions unsupported by empirical evidence. Thinking critically involves our recognizing the assumptions 
underlying our beliefs and behaviours. It means we can give justifications for our ideas and actions. Most 
important, perhaps, it means we try to judge the rationality of these justifications” (p. xviii). In using Brookfield's 
definition, I use the phrase "recognising the assumptions underlying our beliefs and behaviours" in a very applied 
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sense rather than, for example, referring to the assumptions that might underlie one's whole structure of personal 

values. f 

Moving from critical thinking to reflection, we return to Mezirow, who tells us that the three functions ot 
reflection are to guide action, to give coherence to the unfamiliar, and to reassess the justification for what is already 
known. In the R9 context, the latter function is stressed. In relation to reflection, Schon (1 987) quotes Alfred Kyle, 
a Dean of Engineering, as saying, “We know how to teach people to build ships but not how to figure out what ships 
to build” (p. 11). In part, my goal is that, as students learn how to build Dean Kyles's ships, they also learn (and 
practice) a bit about how to figure out which ships to build. I want to see students progressing toward becoming the 
reflective practitioners envisioned by Schon, who think and rethink some of their positions and assumptions, and 

practice what Schon (1983) called "reflection-in-action" (p. 50). 

The second positioning issue relates to whether or not critical thinking can (or should) be taught separately, as a 
general skill applicable across domains of knowledge as advocated by the informal logic movement (ILM) (Ennis, 
1987- Paul 1990; Siegel, 1990). McPeck (1990), contrary to the ILM, insists critical thinking can only be taught in 
the context of a domain’of knowledge. One cannot just think, but has to be thinking about something; and if that 
something were a domain one knows nothing about, critical thinking would be highly unlikely, if not impossible. 
My position here was to take somewhat from both schools of thought although, per McPeck, we are dealing with a 
specific domain of knowledge. I conceptualised three components of critical thinking (as it relates to the R9 
process) as shown below and in poster 1 . 



Figure 1. Conceptualised Components of Critical Thinking and the R9 Process 



Knowledge 



C. Crystallised 
Knowledge 



D. Domain Knowledge 



Skills 



B. Learned Critical Thinking 
and Reflection Skills 



Capability 



A. Basic Reasoning Capability 



Component A, basic reasoning capability, we seem to be bom with. It equates very highly with IQ (McPeck, 
1990), as well as what Cattell referred to as the we might find in a 12 or 13-year old adolescent (Memam & 
Caffarella, 1991). Conponent B, learned critical thinking skills, we are taught - for example in critical thinking 
courses in university. One can find various lists of just what these skills are, and as McPeck noted, the skills may 
vary depending on whether the aim is to win the argument or to seek knowledge and truth. For our purposes, I am 
talking about such skills as Kiersky and Caste (1995) listed: identifying, analysing, and evaluating arguments; 
identifying fallacies; understanding inductive and deductive reasoning; and spotting implications and assumptions. 
For component C, crystallised knowledge, I borrowed the term crystallised from Cattell, since I suggest it equates to 
what Cattell called crystallised intelligence. Roughly, this is intelligence that is grounded in life experience - 
including acquired problem-solving skills and what some might call wisdom. As noted by Cross (in Memam & 
Caffarella, 1991), education and experience heavily influence crystallised intelligence. Finally, component D, 
domain knowledge, represents the course content area. For example, if the course is human resource development, 
we are talking about knowledge in that domain. In terms of figure 1 components, my graduate students typically 
have good reasoning capability (component A) and some crystallised knowledge (component C) including some 
degree of critical thinking ability learned through experience. They may have some knowledge of die domain 
(component D), but few if any have normally had critical thinking courses, per se (component B). The domain 
knowledge component (component D) is our primary course content. The challenge was to include a critical 




P-3 



18 



thinking and reflection component, but as part of the process of teaching the domain knowledge content. My 
response was the R9 process. 

Contribution to HRD Knowledge 

The major contribution of the R9 process is to provide a method to deal with two significant issues for both distance 
and classroom instruction. The first is week-to-week involvement and participation. How can one get everyone 
effectively involved in the class and interacting with the other students? The second issue is matching the structure 
and routine of courses to student needs and expectations, while maintaining a degree of rigor appropriate for 
graduate courses . R9 requires participation and involvement with others. As one of my MBA students put it, 
“There’s no place to hide”. The critical thinking and reflection components require considered, rather than 
spontaneous positions. Simply put, the R9 process involves a sequence of 3 cycles of reading, reflecting, and 
responding related to the material within each teaching/leaming segment. The sequence and content of the steps is 
further described in table 1 and poster 2. 

Developing the Process 

My journey to the R9 Process spanned six courses and about five years. Five of the courses were totally 
asynchronous on-line courses that were part of a distance master’s degree program in instructional and performance 
technology, and one was an MBA course in human resource management that met once a week for two hours. 
Asynchronous courses provide for the exchange of views through a bulletin board-type posting process, but do not 
provide for live, real-time conversation, such as one would have in a chat room or with one of the instant messenger 
services, such as AOL and Microsoft provide. In the MBA course, we had that opportunity for face-to-face 
interaction. The journey was not all about methodology, but was also about my own learning how to be effective in 
a distance environment 

Courses One and Two. Courses one and two were conducted on a DOS-based program called First Reader - 
very basic by today’s standards, but adequate for our purposes. As noted, the students, mostly adults working full 
time, were participating in a distance master’s degree program. I tried to approach the class with adult learning 
principles in mind. This included such things as validation of the participants’ past experiences, recognition that 
they were self-directed learners, involvement in establishing learning objectives, and finalising course design. In 
addition, I tried to get students to share in class governance. Evaluations indicated frustration with what students 
perceived to be a lack of clarity as to what was required of them. In course two, I tightened things up by laying out 
the objectives in advance and taking care of major structure items, but I still tried to provide a lot of room, and let 
discussions go where they would. Since we were more focused, the results were somewhat better, but participation 
and interaction was still not at the level I believed we needed and student evaluations still indicated a desire for more 
structure and clarity. 

Course Three. As a result of my course two experience, I made a major shift with course three by going to a 
high degree of structure. The class was relatively small. I came up with two major discussion questions each week 
and half the class were required to respond to each question. In addition, students were assigned to provide 
feedback to three people who had the other question. I rotated this so that over the semester, everyone provided 
feedback to everyone else more than once. The feedback would then sometimes lead to further discussion but from 
an assessment standpoint, their requirement was one response and three feedback assignments weekly. I did leam 
that I had to be careful with my comments. If I made a comment on a response, for example, that comment would 
become the school solution and influence the people providing feedback - in effect cutting off their critical thinking 
process. The quality of work was good and student evaluations indicated they really liked knowing exactly what 
was required of them. 

Course Four. In course four we moved to Lotus Learning Space, which provided more technical options, 
including web access, and more functions within the course. I stayed with the approach of assigning questions and 
specific feedback responsibilities. This was a larger class, so I increased the number of discussion questions. In 
addition to responding to an assigned question, each student gave feedback to people who had responded to different 
questions. The only downside here was that the assignment scheduling began to get a bit complex. The scheduling 
becomes a little difficult when one must (1) ensure students aren't providing feedback to people who had the same 
question and (2) ensure that feedback assignments rotate to different people throughout the semester. I adopted a 
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minimalist style, with a focus on ensuring that students were going down the right path, rather than trying to dazzle 
them with brilliant comments. 1 provided individual feedback messages commenting on responses and feedback 
quality, feedback on their individual assignments, and would of course respond to a problem, but not in the depth 
some others did. 1 would more likely point students to where they could find an answer for themselves. Again, I 
was pleased with the work of the students and students’ evaluations of the course were very good. 

Course Five. In course five we moved from Lotus Learning Space to a customised version of Lotus Notes. 
This was a most interesting class. One of the things that had concerned me was that while 1 was getting everyone 
involved and working with everyone else, 1 wasn’t getting the degree of reflection that 1 felt we ought to have in a 
graduate course. Since we didn't have the option of the small graduate seminar, 1 decided to add a requirement to 
the responses and feedback that was called the definitive response. After receiving feedback, the students who had a 
particular question (in this course there were six weekly questions, each assigned to three students) had to develop 
and post a final answer. Since this is a heavy workload - three tasks within a week - 1 varied the format somewhat 
during the semester, and breaks in the routine proved welcome. Since 1 wanted the definitive responses to evidence 
some reflection and perhaps reconsideration, 1 tried to introduce the idea of critically appreciative feedback that 
would help those who had answered the questions expand their thinking. We were getting some good cnt,ca * 
thinking in the responses, but not in the feedback. The feedback tended to be rather vanilla - “1 particularly liked 
this and that nice job overall, keep up the good work”. One of the most interesting things to come out of this course 
was the reaction to trying to get students to be critical in their feedback. The term feedback seemed to represent a 
very value laden concept for some students and most of this group resisted doing anything that was not seen as 
positive and reinforcing. Again, 1 was pleased with the work and 90% of the evaluations were quite high. 

Course Six. Following course five, 1 gave more thought to the critical thinking-reflection issue and came up 
with the R9 process which involves three cycles of reading, reflecting, and responding (see table 1.). A major take- 
away from course five was the use of the term feedback. I dropped it and instead now use the term critique. I stress 
that the students' task, more than anything else, is to help expand the thinking of the person who prepared the 
response, and they can't do this by saying "good job, keep it up". Rather, they must point out what they see as flaws, 
point out other perspectives, and suggest alternative ideas. The process as described was used successfully in my 
next distance course and in course six. 

Course six presented a difference challenge in three ways. First, in adapting a process developed for on-line 
courses to an MBA course that meets once a week for two hours. Second, in adapting to a new platform, WebCT. 
Finally, the class had 37 people with 10 basically permanent groups that stay together through the programme. The 
groups varied in size from 3 to 5, which further complicated scheduling. I dealt with this by assigning questions by 
groups, but still with individual initial responses. Critique responsibilities were also assigned by groups, but were 
still individual work. I rotated the groups for whom one’s own group provided critiques. Rather than posting the 
definitive responses, each group provided a short presentation at the weekly class meeting. They had to point out 
their initial positions, what they learned from critiques, and what their final position was. They also had to mention 
the best critique they received. This was followed by a few minutes for questions and discussion. This was the first 
use of WebCT in our MBA programme. Within a few weeks 1 began to get student comments (both in person and 
via WebCT) that this was a good way of learning. Calling the task following initial responses a critique rather than 
feedback seemed to have the desired effect. While I occasionally had to nudge students about various aspects of 
their critiques, there was no resistance to the concept of critically reviewing responses. The class reacted favourably 
to the use of this process, and I perceived that considerably more real discourse and thinking was going on than in 
the lecture-based couises I had taught previously in the programme. A major difference was in seeing students offer 
considered positions, taken after they had done some thinking and after exposure to other perspectives, rather than 
just spontaneous positions taken and offered in the immediacy of the classroom meeting. Student ratings of the 
course were up from the preceding two years, which had followed a more traditional lecture and discussion format. 
(Note: Since course six, I have used R9 in two other face-to-face courses with excellent results - both in terms of 
student ratings and my perceptions of the thinking and learning that was occurring.) 

The R9 Process 

The basic R9 process activities are shown in table 1 and in poster 2. The intent of the process is to provide a 
structure that takes students beyond just reading for retention and recall. It encourages them to think about the 
assignment contents (through question design), to think about and respond to others’ comments, and then, in the 



P-3 



light of critiques and other responses 
raised by their question. 

Table 1 . The Basic R9 Process Activities 



to reflect upon, rethink, and perhaps revise their response to the original issue 



Step Description 



Read the assignment for basic understanding 

Reflect on the contents as they relate to an assigned question 

Respond to the question and post that response 

Read the responses of others as assigned 

Reflect on the responses of others 

Respond to others’ postings with coaching-type (critically appreciative) critiques that will 
expand their thinking on their question 

Receive the critiques of others relating to one’s own initial response 

Reflect on the responses of others who had the same question and on the critiques that were 

received by all who had that question , 

Revise the response, in collaboration with others who had the question, determine the 

definitive response to the question, and post that response (or prese nt, in a face-to-face class). _ 



1 

2 

3 

4 

5 

6 

7 

8 



Using R9 in Distance Education Courses or On -Campus 

It is important to explain the process clearly at the beginning, so that requirements and expectations are clear 
Many students will not have used such a process before, and in particular, the idea of providing critiques jo each 
other needs explanation. There is an advantage for the on-campus course here, since the course can be taken in o the 
computer lab during the first meeting to practice the response and critique process with some very simple questions. 

It may take a few weeks for a distance education class to shake down. On the other hand, all but new distence 
education students will be used to similar software, and it is primarily the three read, reflect, respond cycle i that 
need to be explained. Examples of how both a typical distance education week and how a typical on-campus course 
might be structured and scheduled are shown in poster 2. 

Application Issues 

The Use of Technology. Technology that effectively handles asynchronous communication in a bulletin board- 
type format is essential for the process, but this does not have to be particularly sophisticated. R9 has been used 
successfully with a graduate-level distance education course using a version of Lotus Notes software and in an on- 
campus situation with three different levels of courses using WebCT software „ vamn i P hv 

1 Flexibility. There is more opportunity for flexibility in the process than one might expect. For examp , y 
opening the weekly assignment sheets early, say the preceding Friday, those who want to do most of their work on 
the weekend can both be contributing to the definitive response from the current week and developing their 

response for the next week. ... r _ 

Scheduling If the process is structured so that during the semester, (a) students will receive critiques from 
everyone in the class, (b) students will work with everyone else in the class in developing definitive responses, and 
(c) Til students will have had responsibility for coordinating collaboration on the definitive responses, then 

SChtd Quesfions The questions are very important in order to promote critical thinking. They should require the 
student to think and perhaps question what the reading has to say, do some analysis, take a position, and support it. 

Most students handle the initial response is sues fairly well. _ , 

Crimes. Critiques tend to be another matter. It may take a few weeks to get students into the flow of 
critically appreciative critiques - trying to help other students expand their thinking and perhaps see other 
nersnectives. The on-campus instructor has an advantage here, with the opportunity for weekly face-to-face 
reinforcement of the type critiques that are needed. Grading. It is important that a significant P ar j °f oourse 
grade be a function of successful participation in the R9 process. Participation in the process needs to be importa 
enough that students know they cannot ignore it. Whether critical thinking and reflection is occurring is at least as 
important, if not more important, than the specific ideas developed. Grading should not inhibit expression in the 

Pr 0 C S e Of the Instructor. Front-end work by the instructor is of course important in all courses, but 
more important when using R9. Considerable preparation of the resources that will be needed in 
platform being used is required. Once class begins, one must be constantly monitoring responses and critiques, and 
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it is very important to say enough that students know the instructor is there. If, for example, marks are going to be 
lost due to a superficial response or critique, the student should be told immediately via pnvate message. The 
primary public place for the instructor to offer comments and critique is in connection with definitive responses, 
whether posted or presented in class. At that time, one can ensure key learning points are covered. 



Strengths and Limitations of the Process 

The R9 process is offered as one answer of how to increase critical thinking and reflection in teaching domain 
knowledge-based courses. It can be a useful tool for increasing critical thinking and reflection and improving the 
quality of dialogue in both distance and face-to-face courses at the graduate level. It is to some degree a 'eve//er m 
that it minimises the ability of particular students to dominate or others to not participate. This has been useful when 
integrating groups of students whose prior education has been in systems where the approach was focussed on 
content recall with other groups of students who have had more exposure to a more critical, questioning approach 
While some students feel the workload is heavy, most seem to appreciate the course being clearly laid out and 
knowing exactly what they have to do. This seems to be very important to degree-seeking adults who are working 

full time and may have families, and whose time for study is limited. „ , ,, . 

On the other hand, there are undoubtedly subject matter areas, as well as levels of students (for example, some 
undergraduate levels) where this approach might not be appropriate. In addition, there will be students who want a 
more free wheeling approach with less structure and who may become frustrated. Also, it is somewhat time- 
intensive for the instructor, particularly as class size increases, and I have not yet discovered a way to adapt it for 

really large classes. 
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